Traditionally, the market for focal-plane array solid state imagers, referred to in the art by their underlying technologies (i.e., CCDs for Charge Coupled Devices or CMOS for Complimentary Metal Oxide Semiconductors) has been driven by the market for video cameras, of which a CCD or CMOS sensor is a main component. Video cameras as used for studio broadcast, electronic news gathering (ENG) and consumer camcorders (where the video camera and tape deck or disk drive recorder are integrated into a single compact package) typically provide a color signal. This signal is typically generated by either of two methods.
The first method is typically applied to professional grade equipment and in the higher-grade of consumer equipment. For this method, the incoming image passes through a video lens and then passes through a dichroic prism cluster. The dichroic prism cluster is an assembly of three prisms, two of which have a dichroic (color-band selecting) coating. The first prism will have a surface that reflects one color band, typically blue, to a first CCD or similar imager, and passes the remainder of the colors. The second prism will have a surface that reflects a second color band, typically green, to a second CCD or similar imager. A third prism will accept the balance of the signal, in this case the red band, and transmit it to a third CCD or similar imager. If there is a coating on the third prism, it is typically a coating to reject out-of-band light to which the CCD may be sensitive but which is considered undesirable, for example infrared light. This coating to reject infrared (IR) light may alternatively be placed at the front of the prism cluster, be incorporated into a protective window or be incorporated into the lens. Note that in this first method, each of the CCDs or similar sensors are deployed with their full color spectral response intact as the color-separation is accomplished in the dichroic prism cluster. Furthermore, each of the sensors acquires image data at its full intrinsic resolution, yielding three full resolution images representing three colors from the scene imaged.
The second method is typically used in lower grade equipment such as lower cost and/or very compact ENG systems and in most consumer-grade camcorder systems. Here a single CCD or similar sensor is used. A color-mosaic filter is integrated with the solid state imager, whereby each picture element (pixel) of the imager array (i.e., the sensor) is covered by a corresponding color-band filter. FIG. 1 shows a diagram of a color-mosaic filter 10 showing exemplary red 12, green 14 and blue 16 color filter elements. Typically, one half of the pixels will be associated with a color-band filter that transmits primarily the green image component, one quarter of the pixels will be associated with a color band filter that transmits primarily the blue image component, and one quarter of the pixels will be associated with a color band filter that transmits primarily the red image component. This type of filter is sometimes referred to as a Bayer filter. This system is biased toward the green spectrum as a deliberate compromise to best satisfy the human observer. Humans are most sensitive to green light and less sensitive to blue and red light. The green component carries the most information regarding human faces to which humans are particularly sensitive. Note that in this method the color-mosaic filter is inseparable from the solid state imager, since the filter is aligned precisely with discreet photosensitive elements in the sensor during the manufacturing process and must remain aligned to perform properly. FIG. 2 shows a schematic diagram of a prior art solid state video sensor with attached color-mosaic filter, showing the sensor 20 with attached color-mosaic filter 22, optional IR blocking filter 24, optics assembly 26 and controller 30 attached to sensor 20 by cable 28.
Secondary industries including machine vision are relatively small users of video sensors and cameras compared to the broadcast, ENG and consumer camcorder industries. As a result, manufacturers typically do not manufacture sensors specifically for these secondary markets and therefore these applications must rely on sensors manufactured for other uses. Machine vision applications often require high speed processing and high sensitivity, high resolution and low cost sensors to be successful. For these reasons and others, machine vision applications typically use monochrome sensors, where image data from the scene imaged is rendered in shades of grey. To achieve monochrome imaging, machine vision applications sometimes use sensors developed for use in three-channel dichroic prism cluster professional equipment. This yields high speed processing since color information does not need to be processed, high sensitivity since the entire sensor is available to sense broad band illumination and high resolution since all of the picture elements or pixels are used. This is not, however, a low cost solution since these sensors typically cost much more than mass produced color-mosaic sensors.
Color-mosaic imagers are desirable to be used for machine vision applications because of their low cost. As a result of the permanently attached color filters, their sensors do not perform as well as monochrome sensors, typically requiring additional processing to eliminate color information, thereby forming a monochrome image. Since these cameras are designed to produce color information, the image data from the sensors is typically transmitted either as three separate (red, green and blue) images or encoded in one of the many color formats available. One result of this processing is to reduce resolution, since spatially separate red, green and blue pixels must be combined to form a single monochrome pixel, thereby reducing spatial resolution. In addition, the color filters reduce the sensitivity of each pixel, requiring longer exposure times and/or more light energy on the scene, neither of which is desirable.
The sensitivity of solid state imagers to IR is well known. Methods and apparatus for simultaneously imaging visible and infrared have been developed. For instance, U.S. Pat. No. 3,806,633, entitled Multispectral Data Sensor and Display System, discusses a dual sensor arrangement with optics to produce an IR image registered with a visible image. U.S. Pat. No. 4,651,001, entitled Visible Infrared Imaging Device with Stacked Cell Structure, seeks to produce simultaneous visible and IR images using a novel sensor architecture. The ability of solid state sensors to acquire both visible and IR data is exploited by U.S. Pat. No. 5,555,464, entitled Red/Near-Infrared Filtering for CCD Cameras, wherein a novel mosaic-color filter is used instead of the standard filter to admit IR light in addition to visible light. These approaches all produce color images in addition to IR images or produce false color images of IR data or IR and visible data combined, all of which tend to work against the machine vision goals of cheap, fast, high resolution processing.
There are now other major markets for CCD, CMOS and similar sensors, such as digital still cameras and cellular mobile phones that use color imagers of the color-mosaic filter type. These developments have led to the availability of sensors of much reduced cost that might be applied to machine vision systems and thereby reduce the cost of such systems. One system for using these low cost sensors to image IR data is shown in U.S. Pat. No. 7,235,775, entitled Infrared Imaging Apparatus for Processing Emissions in Infrared and Visible Spectrum. The system described therein removes the IR filter from the sensor, replacing it with a filter that passes only IR data. The system then relies on the visible light color-mosaic filter to form separate three peaks in the IR response and thereby form a false color image of three spectral bands in the IR wavelength region. This approach fails to address two of the major problems with using a color-mosaic filtered sensor to acquire IR data. First, the system must process three separate images, which is not an improvement over standard visible processing. Secondly, the three false color images acquired have reduced resolution with respect to the potential resolution of the sensor since they are spatially multiplexed.
Therefore it is evident that color-mosaic imagers have not been adaptable to the special requirements associated with machine vision due to the limitations mentioned above. Accordingly, there remains a need for reduced cost sensors that can be applied to machine vision applications by obtaining a panchromatic response from color-mosaic filtered solid state imagers.